The invention relates generally to data communication and signal processing methods and apparatus, and in particular to a method and apparatus for reliably and efficiently communicating sequences of image data over a communications channel, for example, a 56 kilobit per second telephone communications channel.
The transmission of sequences of images, and in particular sequences of naturally occurring images such as those represented by a television signal, has been the subject of a significant amount of investigation. Typically, investigators have relied upon the highly redundant nature of successive images in the sequence and have often modeled the image data as a Markov process with a correlation coefficient close to unity. The three-dimensional Markov model provides a motivation for utilizing differential pulse-code-modulation (DPCM) and transform coding techniques to take account of the interframe redundancy.
By analyzing the nature of typical moving video, it is easy to become convinced that the principal change occurring between successive frames is the inhomogeneous motion of the objects within the frame. It has also been recognized that an accurate apparatus and method of estimating and compensating for this spatially dependent motion enables the construction of an interframe data compression method and apparatus which can have substantially better performance than can be achieved by sending a signal representative merely of the difference between successive frames.
As a result, various motion compensating coding methods and apparatus have been developed. These systems typically are either receiver-based motion compensation systems or transmitter-based motion compensation systems. In the receiver-based motion compensation system, the receiver makes a prediction as to the motion and compensates the previous frame for the expected motion. The transmitter, operating in the same manner, then sends only an error signal describing what must be done at the receiver in order to correct the receiver predicted frame. The error signal is typically coded in order to reduce its bandwidth.
For a transmitter-based motion compensation system, the motion estimation process occurs only at the transmitter. Displacement vectors are generally determined over various regions of the image and this data is then transmitted to the receiver along with an error information data signal. At the receiver the compensation process is performed on the previously coded image first using the motion information provided by the transmitter. The error signal data provided by the transmitter is then added to the thus compensated receiver image in order to maintain picture quality.
A particularly advantageous and efficient transmitter-based motion compensation system is described in co-pending applications U.S. Ser. Nos. 740,897, 740,898, and 740,900, all in the name of Brian L. Hinman, assigned to the assignee of this application, and incorporated herein, in their entirety, by reference. The transmitter-based motion compensation system described there, while performing exceedingly well, encodes the motion compensation signals using a lossy compression encoder and hence can introduce artifacts which are not in the original image. Accordingly, the resulting receiver reconstructed image can be degraded somewhat.
The motion compensation process can provide a bad prediction of the new image for several other reasons. For example, the new image frame may not be able to be described by data which details displacements of the previous image; or there may be newly exposed areas of the image, a deformation of the image objects of the old image, illumination changes, or reflections in the image objects.
In addition, the error estimation procedure may fail to track the motion in the old image; the motion estimate may be applied to a region with non-uniform motion, that is, rotation; and coarse quantization can introduce noise into the previous frame that has been used for the prediction. Thus, if the motion compensated data predictions are used in areas of the image where it performs poorly, artifacts will appear in the receiver reconstructed image, especially at high compression rates. Image detail from the previous image will generate noise in the new image. It is to this problem, wherein a prediction performs poorly, that the present invention is directed.
It is therefore an object of the present invention to transmit sequences of images over a communications channel using relatively low bandwidth, and providing high reliability and fidelity. Other objects of the invention are a motion compensation method and apparatus which reliably provide either an accurate estimate of the displacement of the pixels of a scanned image in a sequence or an alternative compensation function, and an improved motion estimation method and apparatus which enable real-time, reliable, and accurate determination of each new image in an image transmission device.